{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Lab 04: Predicting Text</h1>\n",
    "\n",
    "<h2>CS371: Cognitive Science<br>\n",
    "Bryn Mawr College, Fall 2016<br>\n",
    "Prof. Blank\n",
    "</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the cell below, add your by-line, and delete this cell:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "0b6147a20b4c8833be4da22068cc0af4",
     "grade": true,
     "grade_id": "by-line",
     "locked": false,
     "points": 5,
     "solution": true
    }
   },
   "source": [
    "YOUR ANSWER HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lab, we will explore the capability of a network to be able to predict what comes next in a text.\n",
    "\n",
    "# Network with no predefined inputs\n",
    "\n",
    "First, let's explore a Network with no predefined inputs.\n",
    "\n",
    "In these experiments, we will use the idea of a subclass.\n",
    "\n",
    "We create a DynamicInputsNetwork class based on Network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from conx import Network\n",
    "\n",
    "class DynamicInputsNetwork(Network):\n",
    "    def initialize_inputs(self):\n",
    "        pass\n",
    "        # Do some initialization here\n",
    "        # Shuffle them, if self.inputs and \"shullfe\" is set:\n",
    "        # if self.settings[\"shuffle\"]:\n",
    "        #    self.shuffle_inputs()\n",
    "\n",
    "    def inputs_size(self):\n",
    "        # Return the number of inputs:\n",
    "        return 4\n",
    "\n",
    "    def get_inputs(self, i):\n",
    "        # Return a pattern:\n",
    "        temp = [[0, 0], \n",
    "                [0, 1], \n",
    "                [1, 0], \n",
    "                [1, 1]]\n",
    "        return temp[i]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "net = DynamicInputsNetwork(2, 3, 1)\n",
    "\n",
    "def target_function(inputs):\n",
    "    return [int(bool(inputs[0]) != bool(inputs[1]))]\n",
    "\n",
    "net.target_function = target_function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.train(max_training_epochs=5000,\n",
    "          tolerance=0.3,\n",
    "          epsilon=0.1,\n",
    "          shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the network a few times, and make sure you understand how it works.\n",
    "\n",
    "# No predefined inputs, with memory\n",
    "\n",
    "Now, let's try a SRN network (one with memory):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from conx import SRN\n",
    "\n",
    "class DynamicInputsSRN(SRN):\n",
    "    def initialize_inputs(self):\n",
    "        pass\n",
    "        # Do some initialization here\n",
    "        # Shuffle them, if self.inputs and \"shullfe\" is set:\n",
    "        # if self.settings[\"shuffle\"]:\n",
    "        #    self.shuffle_inputs()\n",
    "\n",
    "    def inputs_size(self):\n",
    "        # Return the number of inputs:\n",
    "        return 8\n",
    "\n",
    "    def get_inputs(self, i):\n",
    "        # Return a pattern:\n",
    "        temp = [0, 0, \n",
    "                0, 1, \n",
    "                1, 0, \n",
    "                1, 1]\n",
    "        return [temp[i]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "net = DynamicInputsSRN(1, 5, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "last = 0\n",
    "def target_function(inputs):\n",
    "    global last\n",
    "    retval = [int(bool(inputs[0]) != bool(last))]\n",
    "    last = inputs[0]\n",
    "    return retval\n",
    "\n",
    "net.target_function = target_function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.train(max_training_epochs=5000,\n",
    "          tolerance=0.3,\n",
    "          epsilon=0.1,\n",
    "          shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test out the above network, and make sure you understand how it works.\n",
    "\n",
    "# Predicting text\n",
    "\n",
    "And now, we explore reading through a text, predicting what letter comes next:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from conx import SRN\n",
    "\n",
    "text = (\"This is a test. Ok. What comes next? Depends? Yes. \" +\n",
    "        \"This is also a way of testing prediction. Ok. \" +\n",
    "        \"This is fine. Need lots of data. Ok?\")\n",
    "\n",
    "letters = list(set([letter for letter in text]))\n",
    "\n",
    "def encode(letter):\n",
    "    index = letters.index(letter)\n",
    "    binary = [0] * len(letters)\n",
    "    binary[index] = 1\n",
    "    return binary\n",
    "\n",
    "patterns = {letter: encode(letter) for letter in letters}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Predict(SRN):\n",
    "    def initialize_inputs(self):\n",
    "        pass\n",
    "\n",
    "    def inputs_size(self):\n",
    "        # Return the number of inputs:\n",
    "        return len(text)\n",
    "\n",
    "    def get_inputs(self, i):\n",
    "        letter = text[i]\n",
    "        return patterns[letter]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "net = Predict(len(encode(\"T\")), 5, len(encode(\"T\")))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def target_function(inputs):\n",
    "    index = net.current_input_index\n",
    "    if index + 1 < len(text):\n",
    "        letter = text[index + 1]\n",
    "    else:\n",
    "        letter = \" \"\n",
    "    return patterns[letter]\n",
    "\n",
    "net.target_function = target_function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.train(max_training_epochs=1000,\n",
    "          report_rate=100,\n",
    "          tolerance=0.3,\n",
    "          epsilon=0.1,\n",
    "          shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net.test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your mission is to train a network on your own text. How well does it do?\n",
    "\n",
    "You can use a trained network to generate text. How?\n",
    "\n",
    "Use your network to generate some text."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Reflections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As per usual, please reflect deeply on this week's lab. What was challenging, easy, or surprising? Connect the topics onto what you already know."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "fd25da50d920439579ae721eee54c9f7",
     "grade": true,
     "grade_id": "reflections",
     "locked": false,
     "points": 50,
     "solution": true
    }
   },
   "source": [
    "YOUR ANSWER HERE"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}